"They were very very nice but just not very good": The interplay between resident-supervisor relationships and assessment in the emergency setting.

Purpose: Clinical supervisors hesitate to report learner weaknesses, a widely documented phenomenon referred to as "failure to fail." They also struggle to discuss weaknesses with learners themselves. Their reluctance to report and discuss learner weaknesses threatens the validity of assessment-of-learning decisions and the effectiveness of assessment for learning. Personal and interpersonal factors have been found to act as barriers to reporting learners' difficulties, but the precise role of the resident-supervisor relationship remains underexplored, specifically in the emergency setting. This study aims to better understand if and how factors related to the resident-supervisor relationship are involved in assessment of and for learning in the emergency setting. Methods: We conducted a qualitative study, using semistructured interviews of 15 clinical supervisors in emergency medicine departments affiliated with our institution. Transcripts were independently coded by three members of the team using an iterative mixed deductive-inductive thematic analysis approach. The team then synthesized the coding and discussed analysis following guidelines for thematic analysis. Results: Participating emergency medicine supervisors valued resident-supervisor relationships built on collaboration and trust and believed that such relationships support learning. They described how these relationships influenced assessment of and for learning and how in turn assessment influenced the relationship. Almost all profiles of resident-supervisor relationships in our study could hinder the disclosing of resident weaknesses, through a variety of mechanisms. To protect residents and themselves from the discomfort of disclosing weaknesses and to avoid deteriorating the resident-supervisor relationship, many downplayed or even masked residents' difficulties. Supervisors who described themselves as able to provide negative assessment of and for learning often adopted a more distant or professional stance. Conclusions: This study contributes to a growing literature on failure to fail by confirming the critical impact that the resident-supervisor relationship has on the willingness and ability of emergency medicine supervisors to play their part as assessors.

[Feedback Guide for direct observation of family medicine residents in Canada: a francophone tool]. / Développement et mise à l'essai du Guide de rétroaction francophone pour l'observation directe des résidents en médecine familiale au Canada.

Background: There is no CanMEDS-FM-based milestone tool to guide feedback during direct observation (DO). We have developed a guide to support documentation of feedback for direct observation (DO) in Canadian family medicine (FM) programs. Methods: The Guide was designed in three phases with the collaboration of five Canadian FM programs with at least a French-speaking teaching site: 1) literature review and needs assessment; 2) development of the DO Feedback Guide; 3) testing the Guide in a video simulation context with qualitative content analysis. Results: Phase 1 demonstrated the need for a narrative guide aimed at 1) specifying mutual expectations according to the resident's level of training and the clinical context, 2) providing the supervisor with tools and structure in his observations 3) to facilitate documentation of feedback. Phase 2 made it possible to develop the Guide, in paper and electronic formats, meeting the needs identified. In phase 3, 15 supervisors used the guide for three levels of residence. The Guide was adjusted following this testing to recall the phases of the clinical encounter that were often forgotten during feedback (before consultation, diagnosis and follow-up), and to suggest types of formulation to be favored (stimulating questions, questions of clarification, reflections). Conclusion: Based on evidence and a collaborative approach, this Guide will equip French-speaking Canadian supervisors and residents performing DO in family medicine.

Psychometric validation of the Laval developmental benchmarks scale for family medicine.

BACKGROUND: With the implementation of competency-based education in family medicine, there is a need for summative end-of-rotation assessments that are criterion-referenced rather than normative. Laval University's family residency program therefore developed the Laval Developmental Benchmarks Scale for Family Medicine (DBS-FM), based on competency milestones. This psychometric validation study investigates its internal structure and its relation to another variable, two sources of validity evidence. METHODS: We used assessment data from a cohort of residents (n = 1432 assessments) and the Rasch Rating Scale Model to investigate its reliability, dimensionality, rating scale functioning, targeting of items to residents' competency levels, biases (differential item functioning), items hierarchy (adequacy of milestones ordering), and score responsiveness. Convergent validity was estimated by its correlation with the clinical rotation decision (pass, in difficulty/fail). RESULTS: The DBS-FM can be considered as a unidimensional scale with good reliability for non-extreme scores (.83). The correlation between expected and empirical items hierarchies was of .78, p < .0001.Year 2 residents achieved higher scores than year 1 residents. It was associated with the clinical rotation decision. CONCLUSION: Advancing its validation, this study found that the DBS-FM has a sound internal structure and demonstrates convergent validity.

Analysis of Supervisors' Feedback to Residents on Communicator, Collaborator, and Professional Roles During Case Discussions.

BACKGROUND: Literature examining the feedback supervisors give to residents during case discussions in the realms of communication, collaboration, and professional roles (intrinsic roles) focuses on analyses of written feedback and self-reporting. OBJECTIVES: We quantified how much of the supervisors' verbal feedback time targeted residents' intrinsic roles and how well feedback time was aligned with the role targeted by each case. We analyzed the educational goals of this feedback. We assessed whether feedback content differed depending on whether the residents implied or explicitly expressed a need for particular feedback. METHODS: This was a mixed-methods study conducted from 2017 to 2019. We created scripted cases for radiology and internal medicine residents to present to supervisors, then analyzed the feedback given both qualitatively and quantitatively. The cases were designed to highlight the CanMEDS intrinsic roles of communicator, collaborator, and professional. RESULTS: Radiologists (n = 15) spent 22% of case discussions providing feedback on intrinsic roles (48% aligned): 28% when the case targeted the communicator role, 14% for collaborator, and 27% for professional. Internists (n = 15) spent 70% of discussions on intrinsic roles (56% aligned): 66% for communicator, 73% for collaborator, and 72% for professional. Radiologists' goals were to offer advice (66%), reflections (21%), and agreements (7%). Internists offered advice (41%), reflections (40%), and clarifying questions (10%). We saw no consistent effects when residents explicitly requested feedback on an intrinsic role. CONCLUSIONS: Case discussions represent frequent opportunities for substantial feedback on intrinsic roles, largely aligned with the clinical case. Supervisors predominantly offered monologues of advice and agreements.

User-centered and theory-based design of a professional training program on shared decision-making with older adults living with neurocognitive disorders: a mixed-methods study.

BACKGROUND: We know little about the best approaches to design training for healthcare professionals. We thus studied how user-centered and theory-based design contribute to the development of a distance learning program for professionals, to increase their shared decision-making (SDM) with older adults living with neurocognitive disorders and their caregivers. METHODS: In this mixed-methods study, healthcare professionals who worked in family medicine clinics and homecare services evaluated a training program in a user-centered approach with several iterative phases of quantitative and qualitative evaluation, each followed by modifications. The program comprised an e-learning activity and five evidence summaries. A subsample assessed the e-learning activity during semi-structured think-aloud sessions. A second subsample assessed the evidence summaries they received by email. All participants completed a theory-based questionnaire to assess their intention to adopt SDM. Descriptive statistical analyses and qualitative thematic analyses were integrated at each round to prioritize training improvements with regard to the determinants most likely to influence participants' intention. RESULTS: Of 106 participants, 98 completed their evaluations of either the e-learning activity or evidence summary (93%). The professions most represented were physicians (60%) and nurses (15%). Professionals valued the e-learning component to gain knowledge on the theory and practice of SDM, and the evidence summaries to apply the knowledge gained through the e-learning activity to diverse clinical contexts. The iterative design process allowed addressing most weaknesses reported. Participants' intentions to adopt SDM and to use the summaries were high at baseline and remained positive as the rounds progressed. Attitude and social influence significantly influenced participants' intention to use the evidence summaries (P < 0.0001). Despite strong intention and the tailoring of tools to users, certain factors external to the training program can still influence the effective use of these tools and the adoption of SDM in practice. CONCLUSIONS: A theory-based and user-centered design approach for continuing professional development interventions on SDM with older adults living with neurocognitive disorders and their caregivers appeared useful to identify the most important determinants of learners' intentions to use SDM in their practice, and validate our initial interpretations of learners' assessments during the subsequent evaluation round.

CanMEDS Competencies in Family Medicine Residents: Can Criterion-Based Assessment Improve the Quality of Teacher Feedback?

This article was migrated. The article was marked as recommended. Background: The Université Laval family medicine program has developed an innovative computerized tool called the criterion-based Competency Assessment Tool (CAT), currently undergoing validity assessment. Methods: This study followed a qualitative design assessing written comments collected in the assessment reports from the cohorts before and after the implementation of the CAT (n pre = 200, n post = 200) in order to ascertain the tool's consequence validity. A deductive thematic content analysis was performed and pre- and post-implementation cohorts were compared. Findings: Overall feedback quality does not appear to have changed between cohorts. When analyzing CanMEDS roles separately, each is covered more often, but related comments appear to be less specific. The new report also seems to enable the teacher to tell more with the same number of words. Conclusions: Perhaps since the items are complete, exhaustive, and detailed enough to be self-explanatory, the tool helps the teacher to cover a wider area of competencies without the need to add many details with narrative comments. Consequence validity does not seem to have been substantially affected by changes in the family medicine resident's competency assessment, but the results do not support the contention that comment quality has improved either.

Professional training on shared decision making with older adults living with neurocognitive disorders: a mixed-methods implementation study.

BACKGROUND: Shared decision making with older adults living with neurocognitive disorders is challenging for primary healthcare professionals. We studied the implementation of a professional training program featuring an e-learning activity on shared decision making and five Decision Boxes on the care of people with neurocognitive disorders, and measured the program's effects. METHODS: In this mixed-methods study, we recruited healthcare professionals in family medicine clinics and homecare settings in the Quebec City area (Canada). The professionals signed up for training as a continuing professional development activity and answered an online survey before and after training to assess their knowledge, and intention to adopt shared decision making. We recorded healthcare professionals' access to each training component, and conducted telephone interviews with a purposeful sample of extreme cases: half had completed training and the other half had not. We performed bivariate analyses with the survey data and a thematic qualitative analysis of the interviews, as per the theory of planned behaviour. RESULTS: Of the 47 participating healthcare professionals, 31 (66%) completed at least one training component. Several factors restricted participation, including lack of time, training fragmentation into several components, poor adaptation of training to specific professions, and technical/logistical barriers. Ease of access, ease of use, the usefulness of training content and the availability of training credits fostered participation. Training allowed Healthcare professionals to improve their knowledge about risk communication (p = 0.02), and their awareness of the options (P = 0.011). Professionals' intention to adopt shared decision making was high before training (mean ± SD = 5.88 ± 0.99, scale from 1 to 7, with 7 high) and remained high thereafter (5.94 ± 0.9). CONCLUSIONS: The results of this study will allow modifying the training program to improve participation rates and, ultimately, uptake of meaningful shared decision making with patients living with neurocognitive disorders.

Le nouveau programme formel de mentorat du Collège québécois des médecins de famille.

PROBLÈME À L'ÉTUDE: Les stress inhérents à l'entrée en pratique peuvent décourager les futurs médecins de choisir la médecine de famille. Le Québec fait face à des problèmes de recrutement de médecins de famille en clinique pour remplacer ceux qui s'acheminent vers la retraite. Le mentorat s'avère être un moyen prometteur de soutenir la relève en médecine de famille. OBJECTIF DU PROGRAMME: Aider les médecins de famille en début de pratique à composer de manière satisfaisante avec les enjeux et les défis inhérents à l'insertion professionnelle et à leur développement personnel et professionnel. DESCRIPTION DU PROGRAMME: D'une durée de 12 mois, il se caractérise par des dyades mentoré-mentor, à l'initiative des mentorés; des moyens de soutien aux mentors, dont une communauté de pratique; des modalités de rencontres favorisant la participation de mentorés et de mentors provenant de diverses régions géographiques du Québec. CONCLUSION: L'engagement et le soutien des mentorés et des mentors sont des facteurs essentiels de réussite, mais aussi des défis organisationnels pour la pérennité de programmes formels de mentorat auprès des nouveaux médecins de famille du Québec.

Quebec College of Family Physicians' new formal mentorship program.

PROBLEM ADDRESSED: The stresses that arise during the first years of practice can discourage future physicians from choosing family medicine. Quebec is experiencing challenges in recruiting clinical family physicians to replace those nearing retirement. Mentorship is a promising approach that supports future family physicians. OBJECTIVE OF PROGRAM: To help family physicians at the beginning of their practice to adequately cope with the issues and challenges that come with professional integration and their personal and professional development. PROGRAM DESCRIPTION: A 12-month program that pairs mentors with mentees, on the mentee's initiative; it provides supports for mentors (such as a community of practice), and it comprises meeting formats that encourage the participation of mentees and mentors from different geographic regions across Quebec. CONCLUSION: Engaging and supporting mentees and mentors are essential to success, but might also present organizational challenges to sustaining formal mentorship programs for new family physicians in Quebec.

Talking about treatment benefits, harms, and what matters to patients in radiation oncology: an observational study.

BACKGROUND: Shared decision making is associated with improved patient outcomes in radiation oncology. Our study aimed to capture how shared decision-making practices-namely, communicating potential harms and benefits and discussing what matters to patients-occur in usual care. METHODS: We invited a convenience sample of clinicians and patients in a radiation oncology clinic to participate in a mixed methods study. Prior to consultations, clinicians and patients completed self-administered questionnaires. We audio-recorded consultations and conducted qualitative content analysis. Patients completed a questionnaire immediately post-consultation about their recall and perceptions. RESULTS: 11 radiation oncologists, 4 residents, 14 nurses, and 40 patients (55% men; mean age 64, standard deviation or SD 9) participated. Patients had a variety of cancers; 30% had been referred for palliative radiotherapy. During consultations (mean length 45 min, SD 16), clinicians presented a median of 8 potential harms (interquartile range 6-11), using quantitative estimates 17% of the time. Patients recalled significantly fewer harms (median recall 2, interquartile range 0-3, t(38) = 9.3, p < .001). Better recall was associated with discussing potential harms with a nurse after seeing the physician (odds ratio 7.5, 95% confidence interval 1.3-67.0, p = .04.) Clinicians initiated 63% of discussions of harms and benefits while patients and families initiated 69% of discussions about values and preferences (Chi-squared(1) = 37.8, p < .001). 56% of patients reported their clinician asked what mattered to them. CONCLUSIONS: Radiation oncology clinics may wish to use interprofessional care and initiate more discussions about what matters to patients to heed Jain's (2014) reminder that, "a patient isn't a disease with a body attached but a life into which a disease has intruded."

How often do both core competencies of shared decision making occur in family medicine teaching clinics?

OBJECTIVE: To assess how often risk communication and values clarification occur in routine family medicine practice and to explore factors associated with their occurrence. DESIGN: Qualitative and quantitative cross-sectional study. SETTING: Five university-affiliated family medicine teaching clinics across Quebec. PARTICIPANTS: Seventy-one health professionals (55% physicians, 35% residents, 10% nurses or dietitians) and 238 patients (76% women; age range 16 to 82 years old). MAIN OUTCOME MEASURES: The presence or absence of risk communication and values clarification during visits in which decisions were made was determined. Factors associated with the primary outcome (both competencies together) were identified. The OPTION5 (observing patient involvement in decision making) instrument was used to validate the dichotomous outcome. RESULTS: The presence of risk communication and values clarification during visits was associated with OPTION5 scores (area under the curve of 0.80, 95% CI 0.75 to 0.86, P < .001). Both core competencies of shared decision making occurred in 150 of 238 (63%) visits (95% CI 54% to 70%). Such an occurrence was more likely when the visit included discussion about beginning something new, treatment options, or postponing a decision, as well as when health professionals preferred a collaborative decision-making style and when the visit included more decisions or was longer. Alone, risk communication occurred in 203 of 238 (85%) visits (95% CI 82% to 96%) and values clarification in 162 of 238 (68%) visits (95% CI 61% to 75%). CONCLUSION: Health professionals in family medicine are making an effort to engage patients in shared decision making in routine daily practice, especially when there is time to do so. The greatest potential for improvement might lie in values clarification; that is, discussing what matters to patients and families.

The Impact of a National Competency-Based Medical Education Initiative in Family Medicine.

PURPOSE: Triple C is the Canadian competency-based medical education (CBME) initiative for family medicine. The authors report on a study exploring the impacts Triple C has had across Canada. METHOD: A multi-institutional team conducted a realist study to explore the impact of Triple C implementation in different programs across Canada. Data were collected between March and June 2016 from interviews and focus groups with key medical school stakeholders. Data were analyzed using thematic and template analysis techniques. RESULTS: Data were acquired from 16 of the 17 Canadian medical schools from a combination of program leaders, educators, and trainees. Triple C was implemented in different ways and to different extents depending on context. Newer sites tended to have a more comprehensive implementation than established sites. Urban sites afforded different opportunities to implement Triple C from those afforded by rural sites. Although it was too early to assess its impact on the quality of graduating residents, Triple C was seen as having had a positive impact on identifying and remediating failing learners and on energizing and legitimizing the educational mission in family medicine. Negative impacts included greater costs and tensions in the relationships with other specialties. A principles-based approach to CBME offered flexibility to programs to incorporate variation in their interpretation and implementation of Triple C. Although there was a degree of normalization of practice, it was not standardized across sites or programs. CONCLUSIONS: Triple C has been successfully implemented across Canada but in differing ways and with different impacts.

Validation of the 5-item doctor-patient communication competency instrument for medical students (DPCC-MS) using two years of assessment data.

BACKGROUND: Medical students on clinical rotations have to be assessed on several competencies at the end of each clinical rotation, pointing to the need for short, reliable, and valid assessment instruments of each competency. Doctor patient communication is a central competency targeted by medical schools however, there are no published short (i.e. less than 10 items), reliable and valid instruments to assess doctor-patient communication competency. The Faculty of Medicine of Laval University recently developed a 5-item Doctor-Patient Communication Competency instrument for Medical Students (DPCC-MS), based on the Patient Centered Clinical Method conceptual framework, which provides a global summative end-of-rotation assessment of doctor-patient communication. We conducted a psychometric validation of this instrument and present validity evidence based on the response process, internal structure and relation to other variables using two years of assessment data. METHODS: We conducted the study in two phases. In phase 1, we drew on 4991 student DPCC-MS assessments (two years). We conducted descriptive statistics, a confirmatory factor analysis (CFA), and tested the correlation between the DPCC-MS and the Multiple Mini Interviews (MMI) scores. In phase 2, eleven clinical teachers assessed the performance of 35 medical students in an objective structured clinical examination station using the DPCC-MS, a 15-item instrument developed by Côté et al. (published in 2001), and a 2-item global assessment. We compared the DPCC-MS to the longer Côté et al. instrument based on internal consistency, coefficient of variation, convergent validity, and inter-rater reliability. RESULTS: Phase 1: Cronbach's alpha was acceptable (.75 and .83). Inter-item correlations were positive and the discrimination index was above .30 for all items. CFA supported a unidimensional structure. DPCC-MS and MMI scores were correlated. Phase 2: The DPCC-MS and the Côté et al. instrument had similar internal consistency and convergent validity, but the DPCC-MS had better inter-rater reliability (mean ICC = .61). CONCLUSIONS: The DPCC-MS provides an internally consistent and valid assessment of medical students' communication with patients.

Task Demands in OSCEs Influence Learning Strategies.

THEORY: Models on pre-assessment learning effects confirmed that task demands stand out among the factors assessors can modify in an assessment to influence learning. However, little is known about which tasks in objective structured clinical examinations (OSCEs) improve students' cognitive and metacognitive processes. Research is needed to support OSCE designs that benefit students' metacognitive strategies when they are studying, reinforcing a hypothesis-driven approach. With that intent, hypothesis-driven physical examination (HDPE) assessments ask students to elicit and interpret findings of the physical exam to reach a diagnosis ("Examine this patient with a painful shoulder to reach a diagnosis"). HYPOTHESES: When studying for HDPE, students will dedicate more time to hypothesis-driven discussions and practice than when studying for a part-task OSCE ("Perform the shoulder exam"). It is expected that the whole-task nature of HDPE will lead to a hypothesis-oriented use of the learning resources, a frequent use of adjustment strategies, and persistence with learning. METHOD: In a mixed-methods study, 40 medical students were randomly paired and filmed while studying together for two hypothetical OSCE stations. Each 25-min study period began with video cues asking to study for either a part-task OSCE or an HDPE. In a crossover design, sequences were randomized for OSCEs and contents (shoulder or spine). Time-on-task for discussions or practice were categorized as "hypothesis-driven" or "sequence of signs and maneuvers." Content analysis of focus group interviews summarized students' perception of learning resources, adjustment strategies, and persistence with learning. RESULTS: When studying for HDPE, students allocate significantly more time for hypothesis-driven discussions and practice. Students use resources contrasting diagnoses and report persistence with learning. When studying for part-task OSCEs, time-on-task is reversed, spent on rehearsing a sequence of signs and maneuvers. CONCLUSIONS: OSCEs with similar contents but different task demands lead to opposite learning strategies regarding how students manage their study time. Measuring pre-assessment effects from a metacognitive perspective provides empirical evidence to redesign assessments for learning.

Five shared decision-making tools in 5 months: use of rapid reviews to develop decision boxes for seniors living with dementia and their caregivers.

BACKGROUND: Decision support tools build upon comprehensive and timely syntheses of literature. Rapid reviews may allow supporting their development by omitting certain components of traditional systematic reviews. We thus aimed to describe a rapid review approach underlying the development of decision support tools, i.e., five decision boxes (DB) for shared decision-making between seniors living with dementia, their caregivers, and healthcare providers. METHOD: We included studies based on PICO questions (Participant, Intervention, Comparison, Outcome) describing each of the five specific decision. We gave priority to higher quality evidence (e.g., systematic reviews). For each DB, we first identified secondary sources of literature, namely, clinical summaries, clinical practice guidelines, and systematic reviews. After an initial extraction, we searched for primary studies in academic databases and grey literature to fill gaps in evidence. We extracted study designs, sample sizes, populations, and probabilities of benefits/harms of the health options. A single reviewer conducted the literature search and study selection. The data extracted by one reviewer was verified by a second experienced reviewer. Two reviewers assessed the quality of the evidence. We converted all probabilities into absolute risks for ease of understanding. Two to five experts validated the content of each DB. We conducted descriptive statistical analyses on the review processes and resources required. RESULTS: The approach allowed screening of a limited number of references (range: 104 to 406/review). For each review, we included 15 to 26 studies, 2 to 10 health options, 11 to 62 health outcomes and we conducted 9 to 47 quality assessments. A team of ten reviewers with varying levels of expertise was supported at specific steps by an information specialist, a biostatistician, and a graphic designer. The time required to complete a rapid review varied from 7 to 31 weeks per review (mean ± SD, 19 ± 10 weeks). Data extraction required the most time (8 ± 6.8 weeks). The average estimated cost of a rapid review was C$11,646 (SD = C$10,914). CONCLUSIONS: This approach enabled the development of clinical tools more rapidly than with a traditional systematic review. Future studies should evaluate the applicability of this approach to other teams/tools.

Content and conceptual frameworks of psychology and social work preceptor feedback related to the educational requests of family medicine residents.

OBJECTIVES: Supervision of communication competency in clinical settings in medicine is an important component of professional training. The purpose of this study was to describe the content and rationale of psychology and social work preceptor feedback to family medicine residents who express educational needs during case-based written vignettes. METHODS: We conducted a qualitative study with 25 psychology and social work preceptors from family medicine departments of the three French-speaking universities in the province of Quebec, Canada. During an individual interview, preceptors were asked to respond to three short case-based written vignettes depicting resident educational issues regarding communication and to explain their responses. Authors analyzed the content of responses and the conceptual frameworks reported. RESULTS: The three vignettes elicited 475 responses, including 58 distinct responses and 33 distinct conceptual frameworks. Therapeutic alliance and stages of grief were the two most reported conceptual frameworks. CONCLUSION: The vignettes stimulated a wealth of responses and conceptual frameworks among psychology and social work preceptors in family medicine. PRACTICAL IMPLICATIONS: The complete list of responses could be useful for faculty development activities by stimulating preceptors' reflexive practice with regard to their responses, the educational goals of these responses and the conceptual frameworks underlying their feedback.